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1.0  EXECUTIVE  SUMMARY 

The  goal  of  this  project  was  to  advance  our  understanding  of  the  mental  model  convergence 
process.  Five  technical  objectives  were  established  to  achieve  this  goal.  Two  phases  of 
experimentation  were  undertaken  to  address  the  objectives.  The  first  was  a  quasi-experiment 
where  teams  of  students  undertaking  semester-long  projects  completed  questionnaires  at  four 
time  periods  during  the  semester.  The  second  were  laboratory  behavioral  simulations  where 
teams  of  three  students  completed  two  simulation  sessions  on  progressively  difficult  tasks.  The 
objectives  and  the  corresponding  findings  are  summarized. 

First,  a  model  of  mental  model  convergence  was  developed  by  integrating  existing  theory  from 
literatures  such  as  shared  mental  models,  project  teams,  group  development,  information 
processing,  information  sharing,  and  transactive  memory.  This  model  depicts  a  three-phase 
approach  to  mental  model  convergence.  Specifically,  team  members  (1)  orient  themselves  to  the 
team  and  its  task,  (2)  differentiate  their  own  personal  mental  models  from  the  mental  models  of 
their  team  mates,  and  (3)  integrate  these  differing  perspectives. 

The  second  objective  was  to  explore  the  way  in  which  individually-held  mental  models  converge 
among  team  members  to  become  shared.  The  students  in  the  quasi-experiment  provided 
responses  to  sentence  stems  regarding  a  set  of  teamwork  mental  models  (e.g.,  “The  goals  of  our 
project  are  .  .  .”).  These  responses  were  analyzed  to  ascertain  the  commonality  among  team 
members.  By  comparing  the  results  across  time  periods,  increased  convergence  over  time  was 
uncovered.  The  transcripts  from  the  laboratory  sessions  are  being  coded  to  identify  patterns 
associated  with  the  mental  model  convergence  process. 

Three  methods  were  examined  for  measuring  mental  model  convergence  to  address  the  third 
objective.  The  first  two  were  described  in  the  previous  paragraph.  The  third  was  examined 
using  data  collected  at  the  end  of  the  simulation.  Team  members  completed  questionnaires 
comprised  of  existing  scales  designed  to  measure  team  perceptions  about  constructs  such  as  goal 
clarity,  cooperation,  and  team  skills.  The  interrater  agreement  among  member  responses  was 
used  to  score  convergence  in  team  members’  mental  models. 

The  fourth  and  fifth  objectives  were  to  confirm  that  multiple  mental  models  function 
simultaneously  and  to  determine  how  shared  mental  models  regarding  teamwork  impact  team 
performance.  The  quasi -experiment  results  indicate  that  team  members  who  have  a  common 
understanding  at  the  beginning  of  the  project  of  their  team’s  goals  and  how  they  are  going  to 
interact  and  at  the  end  of  the  project  understand  who  is  doing  what  task  perform  better.  Using 
the  behavioral  simulation  data,  team  performance  was  regressed  on  the  teamwork  mental  model 
convergence  scores.  The  results  confirm  that  multiple  mental  models  function  simultaneously 
because  the  mental  models  tested  had  unique  relationships  with  performance.  The  results  also 
reveal  relationships  between  the  mental  models  and  performance.  From  the  first  session,  the 
goal  acceptance,  cooperation  and  coordination  mental  models  were  related  to  time  performance 
and  the  goal  clarity  and  cooperation  mental  models  were  related  to  member  satisfaction.  From 
the  second  session,  goal  clarity  (negative)  and  cooperation  mental  models  were  related  to  time 
performance,  the  team  staffing  quality  mental  model  was  related  to  quality  performance,  and  the 
cooperation  mental  model  aided  satisfaction. 
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2.0  PROJECT  SUMMARY 

No  consensus  among  researchers  studying  shared  cognition  exists  regarding  the  identification  of 
what  should  be  shared,  the  measurement  of  shared  mental  models  (SMM),  and  the  effects  of 
SMM  on  team  outcomes  (Cannon-Bowers  &  Salas,  2001).  This  research  project  was  undertaken 
to  address  these  issues  and  extend  our  theoretical  understanding  of  shared  cognition  in  at  least 
three  ways.  First,  minimal  research  has  been  conducted  in  the  area  of  SMM  that  integrates  extant 
research  from  related  fields  (Mohammed  &  Dumville,  2001).  This  research  project  drew  together 
research  on  SMM,  project  teams,  group  development,  information  processing,  information 
sharing,  and  transactive  memory.  Further,  evidence  is  included  from  areas  similar  to  SMM,  such 
as  group  mind  (e.g.,  Weick  &  Roberts,  1993),  interpretive  schema  (e.g.,  Bartunek,  1984; 
Dougherty,  1992),  intersubjectivity  (e.g.,  Eden  et  ah,  1981),  shared  cognition  (e.g.,  Cannon- 
Bowers  &  Salas,  2001),  shared  meaning  (e.g.,  Smircich,  1983),  sociocognition  (e.g.,  Gruenfeld 
&  Hollingshead,  1993),  and  team  mental  models  (e.g.,  Klimoski  &  Mohammed,  1994).  Second, 
in  this  research,  mental  model  convergence  (MMC)  is  modeled  from  an  information  processing 
perspective.  While  other  studies  have  proposed  that  information  processing  can  occur  at  the 
group  level  (e.g.,  Comer  et  ah,  1994;  Gibson,  2001;  Gruenfeld  &  Hollingshead,  1993;  Hinsz  et 
ah,  1997),  none  have  specifically  applied  information  processing  to  teamwork  functions  as  was 
done  in  this  project.  Third,  this  research  extends  our  current  understanding  of  SMM  content  by 
investigating  the  simultaneous  existence  of  multiple  mental  models.  Previous  work  has 
discussed  possible  SMM  content  requirements  (e.g.,  Cannon-Bowers  et  ah,  1993)  and  research 
has  tested  for  the  simultaneous  existence  of  teamwork  and  taskwork  mental  models  (e.g., 
Mathieu  et  ah,  2000;  Smith-Jentsch  et  ah,  2005).  To  my  knowledge,  however,  multiple  teamwork 
mental  models  have  never  been  captured  in  one  study. 

2.1.  Goals 

The  long  term  goal  of  this  research  is  to  advance  the  theoretical  understanding  of  shared 
cognition  in  project  teams  by  (1)  integrating  existing  theory  in  related  fields  to  establish  a  model 
of  mental  model  convergence;  and  (2)  validating  the  model  through  experimentation. 

2.2.  Technical  Objectives 

This  project  has  five  objectives:  (1)  to  devise  a  model  of  mental  model  convergence;  (2)  to 
explore  the  way  in  which  individually-held  mental  models  converge  among  team  members  to 
become  shared;  (3)  to  examine  different  means  of  measuring  mental  model  convergence;  (4)  to 
distinguish  among  the  multiple  mental  models  working  simultaneously  within  a  team;  and  (5)  to 
determine  how  shared  mental  models  regarding  teamwork  impact  team  performance. 

2.3.  Approach 

Two  phases  of  experimentation  were  undertaken:  (1)  a  longitudinal  quasi-experiment  to  test  for 
the  convergence  of  individually-held  mental  models  over  time;  (2)  a  set  of  controlled  behavioral 
simulations  to  examine  the  effects  of  communication  medium  (FF/CM),  session  (time  1/time  2), 
and  condition  (unrestricted/time  pressure/environmental  uncertainty)  on  mental  model 
convergence  and  team  performance. 
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3.0  CONCEPTUALIZATION 

The  conceptualization  of 
mental  model  convergence 
(MMC)  (shown  in  Figure  1) 
underlying  the  research 
conducted  for  this  proposal 
achieves  the  first  technical 
objective:  to  devise  a  model 
of  MMC.  The  model  is 
focused  on  developing 
teamwork  SMM  because 
successful  teams  put 
significant  effort  into 
developing  protocols  for 
working  together  as  a  team, 
such  as  an  agreed  upon 
approach  for  conducting  the 
requisite  work  (Katzenbach 
&  Smith  1999).  Ensuring 
that  team  members  explicitly 
focus  on  teamwork  at  the 
onset  of  team  activity,  as 
suggested  in  Figure  1,  is 
critical  for  at  least  two 
reasons.  First,  many 
organizations  provide  the 
most  support  for  team 
development  early  in  the 
team’s  life  cycle  (Druskat  &  Pescosolido  2002).  Second,  team  members  will  develop  protocols 
(implicitly  or  explicitly)  for  collective  work  (i.e.,  the  teamwork  phase  of  the  team’s  life  cycle) 
(Smircich  1983);  moreover,  they  will  develop  them  very  quickly  and  sustain  them  for  extended 
periods  of  time  (Gersick  1988).  Even  when  MMC  occurs  implicitly  as  individual  team  members 
actively  consider  teamwork  issues  relevant  to  their  own  personal  agendas,  the  phases  proposed 
will  still  be  relevant.  The  results,  however,  will  be  inconsistent  across  team  members.  Further, 
the  accuracy  and  comprehensiveness  of  the  resultant  SMM  may  be  questionable.  Under  these 
circumstances,  team  members  need  to  actively  think  about  teamwork  at  the  beginning  of  their 
life  cycle.  Thus,  this  model  advocates  that  teams  begin  their  life  cycle  by  focusing  on  teamwork. 

During  this  focus  period,  team  members  develop  shared  mental  models  (SMM)  by  proceeding 
through  the  three  phases  of  MMC:  orientation,  differentiation,  and  integration.  These  three 
phases  stem  from  group  development  and  information  processing  research.  Whether  the  phases 
of  group  development  are  called  forming,  storming,  and  norming  (Tuckman  1965);  team  finding, 
designing,  and  transforming  (Uhl-Bien  &  Graen  1992);  or  some  other  variation  found  in  the 
literature;  the  generalized  process  is  basically  the  same  (Tuckman  &  Jensen  1977).  Members:  (1) 
orient  themselves  to  their  unique  domain;  (2)  create  their  own  view  of  the  situation,  which  may 
or  may  not  be  similar  to  their  fellow  team  members’  views;  and  (3)  allow  their  individual 
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perspective  to  evolve  into  a  team  view.  Likewise,  information  processing  occurs  when 
individuals  differentiate  among  available  alternatives  and  subsequently  reconcile,  or  integrate, 
similarities  and  differences  among  the  alternatives  to  determine  a  course  of  action  (Driver  & 
Streufert  1969;  Schroder  et  al.  1967). 

This  model  exploits  the  synergies  between  group  development  and  information  processing  and 
represents  a  bottom-up,  or  emergent,  process  (Kozlowski  &  Klein  2000).  The  process  begins  at 
the  onset  of  team  activity  when  each  individual  team  member  has  a  unique,  independent  view  of 
the  team,  its  assignment,  and  its  context.  The  team  members  then  begin  to  orient  themselves  to 
the  team  situation.  Orientation  is  retained  from  the  research  on  group  development  to  represent 
the  phase  where  the  team  members  collect  information  about  their  unique  domain  that  will  be 
used  to  create  SMM.  Differentiation  and  integration  stem  from  information  processing  (Driver  & 
Streufert  1969;  Schroder  et  al.  1967).  Differentiation  occurs  as  team  members  interpret  their 
situation  in  the  second  phase  of  group  development.  From  an  information  processing  perspective, 
interpretation  occurs  as  team  members  sort  through  information  they  have  collected.  This  sorting 
process  allows  them  to  differentiate  among  their  fellow  team  members’  knowledge  and  beliefs.  It 
also  represents  the  phase  of  MMC  where  team  members  shift  from  individual-level  mental 
models  to  team-level  mental  models.  As  a  result,  the  mental  models  held  by  individual  team 
members  depict  the  differences  among  team  members’  knowledge  and  beliefs  about  the  team 
and  its  task. 

In  the  integration  phase  of  MMC,  the  individual  team  members’  mental  models  remain  at  the 
team  level,  but  the  differences  sorted  out  during  differentiation  are  transformed  into  a  converged 
representation  of  the  collective  views  of  the  team  members.  Thus,  integration  can  be  viewed  as  a 
transformational  process  (Dansereau  et  al,  1999).  Specifically,  this  process  takes  place  as  the 
team  members  integrate  their  perspectives  of  the  team  by  identifying  and  strengthening  the 
interrelationships  among  themselves  in  order  to  achieve  unity  of  effort.  I  do  not  intend  to  imply 
that  all  team  members  will  hold  an  identical  set  of  mental  models  at  the  conclusion  of  the 
integration  phase.  Rather,  the  degree  of  integration  (i.e.,  the  strength  of  the  interrelationships  that 
are  developed  among  team  members)  must  be  carefully  considered. 

As  shown  in  Figure  1,  multiple  mental  models  can  be  developed  simultaneously,  although  the 
progression  may  occur  at  different  speeds  for  each  one.  The  speed  may  be  dependent,  in  part, 
upon  the  amount  of  previously  held  knowledge  team  members  possess  that  is  applicable  to  the 
current  situation.  For  example,  if  a  majority  of  team  members  have  worked  together  on  previous 
projects,  logic  suggests  that  the  time  required  for  attaining  MMC  will  be  much  shorter  than  the 
time  required  for  a  set  of  individuals  with  no  previous  experience  working  together. 
Additionally,  the  information  processed  for  one  mental  model  may  influence  the  development 
cycle  of  others.  As  such,  while  the  development  phases  are  depicted  as  occurring  linearly,  new 
information  attained  regarding  one  mental  model  may  have  ramifications  for  other  mental 
models.  The  team  may  need  to  regress  to  an  earlier  phase  and  revise  the  affected  mental 
model(s),  accordingly.  The  dotted  arrows  in  Figure  1  allow  for  this  iterative  process  to  occur. 
Upon  creation  of  teamwork  SMM,  team  members  then  shift  their  focus  to  their  assigned 
taskwork  until  a  time  when  one  or  more  teamwork  mental  models  need  revision.  The  decision 
point  in  Figure  1  represents  the  provision  for  revising  teamwork  mental  models.  Taskwork  may 
also  require  a  unique  set  of  SMM,  but  was  not  the  focus  of  the  present  study. 
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4.0  EXPERIMENTATION 


4.1.  Quasi-Experiment 

4.1.1.  Technical  Objectives 

The  quasi-experiment  addressed  the  following  technical  objectives  (from  section  2.2): 

(2)  to  explore  the  way  in  which  individually-held  mental  models  converge  among  team  members 
to  become  shared 

(3)  to  examine  different  means  of  measuring  mental  model  convergence 

(4)  to  distinguish  among  the  multiple  mental  models  working  simultaneously  within  a  team 

(5)  to  determine  how  shared  mental  models  regarding  teamwork  impact  team  performance 

4.1.2.  Data  Collection 

One-hundred  two  undergraduate  business  students  from  the  University  of  Massachusetts 
Isenberg  School  of  Management  participated  in  the  quasi-experiment.  These  students  were 
enrolled  in  one  of  two  classes  that  required  the  completion  of  a  team  semester-long  project.  The 
students  were  allowed  to  self-select  their  teams  in  both  classes.  Twenty-five  teams  with  three  to 
five  members  each  comprise  the  sample.  Team  members  earned  extra  credit  for  participating  in 
the  study  if  they  completed  all  questionnaires  throughout  the  semester.  If  all  team  members  from 
the  team  completed  all  questionnaires,  then  the  team  was  entered  in  a  random  drawing  where 
they  had  an  opportunity  to  win  $50  per  member.  The  odds  of  winning  the  prize  money  were  1:5. 

At  four  points  during  the  semester,  the  team  members  completed  a  questionnaire.  Time  1  was 
immediately  after  the  semester  project  was  introduced  and  Time  2  was  one  week  later.  Time  3 
and  Time  4  questionnaires  were  administered  at  mid-semester  and  at  the  end  of  the  semester, 
respectively.  Questionnaires  were  used  to  collect  data,  because  in  circumstances  where  mental 
models  can  be  manipulated  explicitly,  i.e.,  subjects  are  aware  that  they  must  manipulate  their 
mental  models  in  order  to  complete  their  assigned  tasks,  they  can  be  appropriately  described  via 
verbalization  methods  such  as  questionnaires  (Rouse  and  Morris,  1986). 

To  assess  knowledge  similarity  of  the  team  member  mental  model  content,  questionnaire  items 
were  written  based  on  the  sentence  completion  test  developed  in  the  field  of  integrative 
complexity  (Schroder  et  al.,  1967).  Respondents  were  asked  to  complete  sentence  stems  related 
to  the  mental  models  of  interest.  Sentence  stems  were  written  to  identify  individual  perceptions 
regarding  the  project  goals  (“The  goals  of  our  project  are  .  .  .”),  team  processes  (“Our  team  has 
discussed  coordinating  workflow  among  team  members  in  the  following  manner  .  .  .”),  team 
communication  mediums  (“Our  team  has  discussed  using  the  following  modes  of 
communication  to  exchange  information  .  .  .”),  and  team  organization  structure  (“Our  project 
team  is  organized  (i.e.,  who  is  the  leader,  etc.)  in  the  following  manner  .  .  .”).  These  sentence 
stems  required  respondents  to  focus  on  directly  observable  behaviors  rather  than  generalizations 
drawn  from  espoused  theory,  following  Argyris  and  Schdn’s  (1974)  methodology,  to  overcome 
the  inherent  difficulties  in  tapping  theories-in-use. 

The  responses  were  content  analyzed.  The  commonality  of  responses  was  determined  and  a 
score  was  assigned  representing  the  degree  of  sharing  found  in  the  team  members’  mental 
models  for  each  sentence  stem.  Scores  were  assigned  based  on  an  Agreement  Index  developed 
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for  this  project  that  is  analogous  to  the  7-point  Integration  Index  devised  by  Schroder  et  al. 
(1967).  Independently,  two  raters  assessed  each  team  member’s  response  for  a  given  team  to 
determine  a  score.  Their  interrater  reliability  was  73%.  When  the  raters  did  not  agree,  they 
discussed  their  respective  scores  and  came  to  an  agreement  on  the  appropriate  score  for  the 
team’s  mental  model  under  consideration. 

4.1.3.  Findings 

The  findings  are  shown  in  Figures  2  and  3.  In  Figure  2,  the  rater’s  scores  for  the  entire  sample’s 
mental  models  are  plotted.  The  significant  shifts  in  level  of  agreement  are  detailed.  In  Figure  3, 
the  sample  was  split  into  two  groups  based  on  their  team  performance  score  (assessed  as  the 
team’s  grade  on  their  semester-long  project).  Thus,  the  graphs  depict  the  convergence  patterns 
of  high  performing  and  low  performing  teams  for  comparison.  These  results  address  each 
technical  objective  outlined  in  section  4.1.1  as  described  in  the  following  paragraphs. 

The  first  two  objectives  were  to  explore  the  way  in  which  individually-held  mental  models 
converge  among  team  members  to  become  shared  and  to  examine  different  means  of  measuring 
MMC.  The  sentence  completion  approach  proved  to  be  a  useful  method  for  assessing  mental 
model  content.  The  results  in  Figure  2  can  be  used  to  explore  convergence  over  time.  For  team 
processes,  team  organization,  and  team  communication,  the  shift  in  mental  model  content 
converges  significantly  from  the  initial  questionnaire  to  the  second  that  was  completed  one  week 
later.  The  shift  was  not  significant  for  the  team  goals  mental  model.  One  plausible  reason  for 
this  lack  of  significance  may  be  that  the  project  goals  in  both  classes  are  explained  thoroughly 
just  prior  to  when  the  team  members  complete  the  first  questionnaire  and  the  initial 
understanding  of  the  team’s  purpose  is  well  established  and  commonly  understood. 

The  next  objective  was  to  distinguish  among  the  multiple  mental  models  working  simultaneously 
within  a  team.  As  can  be  seen  in  Figure  2  and  3,  the  patterns  of  convergence  and  the  mental 
models’  relationships  with  team  performance  are  different  across  mental  models.  These  results 
provide  evidence  that  multiple  mental  models  function  simultaneously  and  in  different  ways. 

The  last  objective  was  to  determine  how  shared  mental  models  regarding  teamwork  impact  team 
performance.  The  results  in  Figure  3  show  the  varying  results  for  high  and  low  performing 
teams  for  each  mental  model  studied.  Significant  differences  between  high  and  low  performing 
teams  were  found  as  follows,  where  high  performing  teams  had  more  convergent  mental  model 
content:  goal  mental  model  Time  1,  team  processes  mental  model  Time  1  and  Time  2,  team 
organization  mental  model  Time  3  and  Time  4.  These  results  suggest  that  teams  who  understand 
their  goals  and  how  they  are  going  to  interact  at  the  onset  of  the  team’s  life  cycle  and  how 
responsibilities  are  divided  at  the  end  of  their  life  cycle  will  perform  better. 

One  additional  finding  of  interest  relates  to  shifts  in  the  mental  model  content  over  time.  As  new 
information  became  available  to  the  team,  the  mental  model  content  shifted  and  the  level  of 
convergence  remained  high.  For  example,  as  the  semester  progresses,  team  members’  goal 
mental  models  shifted  from  containing  information  about  the  overarching  goal  to  more  detailed 
information  about  the  task  oriented  goals  required  to  complete  the  project.  Similarly,  the  team 
organization  structure  mental  model  went  from  a  high  level  description  of  the  structure  to  a 
detailed  description  of  who  is  doing  what  tasks  needed  to  complete  the  project. 
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Figure  2:  Mental  Model  Convergence  Over  Time 


m  agreement  is  significant  for  Team  Processes,  Team  Organization,  and  Team  Communication  at  the  p<0.05. 


Figure  3:  Mental  Model  Convergence-Performance  Relationship 


Difference  between  High  and  Low  performing  teams  is  No  significant  different  between  High  and  Low  performing 

significant  at  T=3  (p=0.06)  and  T=4  (p=0.08).  teams  over  time. 


4.2.  Laboratory  Behavioral  Simulations 


4.2.1.  Technical  Objectives 

The  laboratory  behavioral  simulations  addressed  the  following  technical  objectives  (from  section 

2.2): 

(2)  to  explore  the  way  in  which  individually-held  mental  models  converge  among  team  members 
to  become  shared 

(3)  to  examine  different  means  of  measuring  mental  model  convergence 

(4)  to  distinguish  among  the  multiple  mental  models  working  simultaneously  within  a  team 

(5)  to  determine  how  shared  mental  models  regarding  teamwork  impact  team  performance 

4.2.2.  Data  Collection 

Two-hundred  sixteen  undergraduate  business  students  from  the  University  of  Massachusetts 
Isenberg  School  of  Management  were  randomly  formed  into  teams  of  three.  The  students 
received  extra  credit  for  their  participation.  Additionally,  the  teams  with  the  highest  performance 
received  a  cash  bonus  of  $50  per  team  member  (odds  of  winning  were  approximately  1 :5). 

The  participants  performed  a  personnel  scheduling  task  adapted  from  Bachrach  et  al.  (2001), 
Earley  (1994),  and  Steele- Johnson  et  al.  (2000).  The  participants  were  asked  to  assign  10 
employees,  each  with  different  hourly  wages  to  a  work  schedule  for  a  hypothetical  organization. 
Workforce  requirements  (i.e.,  number  of  employees  per  shift)  were  given  to  the  team.  To  ensure 
collaboration,  each  team  member  represented  a  different  organizational  function  with  conflicting 
rules  for  schedule  completion,  in  the  sense  that  maximizing  one  function’s  rules  led  to  breaking 
the  others’  rules.  The  functional  roles  and  associated  rules  were:  (1)  human  resources 
representative,  each  employee  can  work  no  more  than  10  hours  per  day  and  50  hours  per  week; 
(2)  union  representative,  each  employee  must  work  four  hours  per  day  if  called  into  work  and  30 
hours  per  week;  (3)  operations  manager,  the  schedule  requirements  must  be  met  exactly.  If  any 
of  the  rules  were  broken  in  the  final  schedule,  a  penalty  was  assessed.  The  task  was  pretested 
using  a  comparable  sample  to  ensure  that  the  students  found  the  exercise  interesting  and  that  the 
level  of  difficulty  was  appropriate  to  the  population  and  time  allotted  for  each  session. 

We  used  a  2  (communication  medium)  x  2  (session)  x  3  (condition)  design.  Three-person  teams 
were  randomly  assigned  to  a  computer-mediated  (CM)  or  face-to-face  (FF)  communication 
medium  to  complete  the  simulated  task.  The  FF  team  members  were  seated  around  a  table  in  a 
private  room.  Each  participant  had  a  worksheet  and  instructions.  Only  one  solution  per  team  was 
accepted.  The  sessions  were  video-recorded;  each  participant  gave  consent  to  the  video¬ 
recording  procedure.  A  digital  timer  placed  on  the  table  recorded  the  time  elapsed  since  the 
session  began.  The  CM  sessions  were  conducted  in  a  laboratory  consisting  of  10  personal 
computers  linked  in  a  local  area  network.  Thus,  three  teams  could  be  simultaneously  completing 
the  behavioral  simulation.  Participants  from  one  team  were  seated  so  no  members  from  the  same 
team  were  next  to  each  other.  Using  this  arrangement,  the  team  members  could  see  each  other 
(since  they  were  located  in  the  same  room),  but  were  far  enough  apart  to  be  unable  to  read  each 
other’s  notes.  Furthermore,  participants  were  not  allowed  to  talk  with  each  other  during  the 
simulation.  The  teams  in  the  CM  condition  communicated  through  a  synchronous  computer 
conferencing  system  specifically  developed  for  the  purposes  of  the  simulation.  The  system 
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consisted  of  a  screen  with  the  task  requirements  (this  view  was  identical  to  the  paper  copy  given 
to  all  participants  in  both  media  conditions),  a  text  messaging  area,  and  a  timer  depicting  the 
elapsed  time  since  the  beginning  of  the  session.  A  unique  simulation  environment  was  created 
for  each  team  that  allowed  them  to  work  synchronously  on  the  schedule.  All  team  members  had 
equal  access  to  the  screen  and  could  assign/delete  workers,  view  any  assignments  or  deletions 
made  by  team  mates,  and  exchange  messages  with  team  mates.  The  communication  was  not 
anonymous;  the  messages  were  identified  by  the  sender’s  name  and  appeared  in  the  order  in 
which  they  were  sent.  The  team  members  could  scroll  through  the  communication  history 
when/if  they  wanted  to  review  past  messages. 

Each  team  gained  experience  working  together  as  a  team  by  completing  two  sessions.  The 
second  session  was  scheduled  at  the  convenience  of  all  team  members,  between  three  and  seven 
days  after  the  first.  The  functional  roles  the  team  members  represented,  and  their  associated 
rules,  remained  the  same  for  the  first  and  second  sessions,  but  the  schedule  requirements 
changed.  In  the  first  session,  employee  requirements  per  day  were  given  (see  Figure  4).  For  the 
second  task,  employee  requirements  per  shift  were  given  (see  Figure  5),  which  made  the  second 
task  more  complex  than  the  first. 

The  conditions  were  unrestricted,  time  pressure,  and  environmental  uncertainty.  The  teams 
assigned  to  the  unrestricted  condition  were  given  unlimited  time  to  complete  the  task  and  were 
unbothered  throughout  the  session.  Under  time  pressure,  teams  were  given  45  and  30  minutes  to 
complete  the  task  during  the  first  and  second  sessions,  respectively.  Environmental  uncertainty 
was  introduced  by  promoting  (first  session)  or  demoting  (second  session)  one  of  the  employees 
available  to  be  scheduled  20  minutes  into  the  session.  The  sample  size  by  condition  is  shown  in 
Table  1. 

Table  1.  Behavioral  simulation  sample  size  by  communication  medium  and  condition 


FF 

CM 

Unrestricted 

36 

36 

Time  Pressure 

30 

31 

Environmental  Uncertainty 

33 

33 

After  study  participants  volunteered  for  the  simulation,  they  were  randomly  assigned  to  teams. 
The  teams  were  informed  about  the  exact  time  when  their  team  was  scheduled.  Upon  arrival,  the 
team  members  were  introduced  and  seated.  The  teams  were  informed  that  the  general  goal  of  the 
simulation  was  to  investigate  how  teams  work  together  and  reminded  that  the  best  performing 
teams  would  receive  a  prize  of  $50  for  each  team  member.  The  teams  were  told  that  the  task  was 
estimated  to  take  about  45  minutes  to  complete,  but  the  quality  of  their  work  schedule 
determined  the  team’s  performance.  The  teams  were  not  pressed  to  finish  within  45  minutes  and 
were  left  to  work  until  all  three  members  were  satisfied  with  the  solution  (except  in  the  time 
pressure  condition).  In  the  CM  condition,  a  brief  training  session  on  the  use  of  the  computer 
software  was  conducted  immediately  after  the  instructions  were  given. 
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WORKFORCE  SCHEDULE  WORKSHEET  SCENARIO  1  Team 


required 
per  shift 


WORKFORCE  SCHEDULE  WORKSHEET  SCENARIO  2  Team 


-  8  pm 


The  teams  read  written  instructions  about  the  task  before  they  started  working  together.  After  the 
task  was  completed  and  the  final  solution  submitted  to  the  experimenter,  each  participant 
individually  completed  a  questionnaire  regarding  the  team  and  its  processes.  Existing  scales 
were  used  when  available  and  were  created  for  this  study  when  no  appropriate  scale  could  be 
identified.  The  measures  included  collective  efficacy  (Riggs  et  al.,  1994),  goal  acceptance 
(Latham  &  Steele,  1983),  team  membership  (based  on  Ganster  &  Dwyer,  1995),  work  allocation 
(devised  for  this  study),  decision  making  (Coopman,  2001),  team  member  skills  (devised  for  this 
study),  interdependence  (Johnson  et  al.,  1983),  goal  clarity  (Weber  &  Weber,  2001),  satisfaction 
(Van  Dyne  &  LePine,  1998),  cooperation  (Campion  et  al.,  1993),  conflict  (Jehn  &  Mannix, 
2001),  coordination  (Denison  et  al.,  1996),  and  cohesion  (modified  from  Seashore,  1954). 

Two  dependent  measures  were  used  to  assess  team  performance  following  the  guidance  of  Straus 
and  McGrath  (1994),  who  identified  quantity,  quality  and  speed  as  the  primary  indicators  of  team 
task  performance.  In  this  experiment  the  teams  were  asked  to  develop  one  workforce  schedule,  in 
approximately  45  minutes.  Thus,  the  quantity  was  not  a  relevant  performance  indicator  in  this 
study.  The  teams  developed  as  many  “practice”  schedules  as  they  wished,  but  were  required  to 
agree  upon  one  final  schedule  per  team  for  submission  at  the  end  of  the  simulation  session.  The 
teams’  schedules  were  assessed  for  quality.  Quality  was  calculated  as  the  cost  associated  with 
the  particular  assignment  developed,  including  penalties  for  any  broken  rules.  Specifically, 
using  each  employee’s  cost  per  hour  and  the  corresponding  number  of  hours  assigned  to  work, 
the  cost  per  employee  was  calculated.  The  cost  for  all  employees  was  added.  Next,  the  schedules 
were  checked  to  see  if  the  rules  given  to  the  teams  were  enforced.  Each  time  a  rule  was  broken,  a 
penalty  of  1.8  times  the  respective  employee  wage  was  added.  Finally,  speed  to  solution  was 
recorded  as  the  time  from  the  moment  the  team  members  started  working  until  the  final  solution 
was  submitted. 

Our  measurement  approach  created  performance  measures  where  better  performance  was  the 
smaller  value.  To  aid  interpretation  of  results  and  follow  the  more  conventional  approach  of 
“more  is  better,”  we  transformed  both  measures.  Quality  performance  reported  is  calculated  as 
the  optimal  assignment  score  divided  by  a  team’s  score.  Thus,  the  closer  a  team’s  score  is  to  the 
optimal  score,  the  closer  their  quality  performance  is  to  1.0  (i.e.,  higher  values  represent  better 
quality  performance).  Time  performance  is  calculated  as  the  slowest  team’s  time  plus  one  minus 
a  team’s  recorded  time.  Thus,  the  slowest  team  (i.e.,  the  team  with  the  worst  time  performance) 
has  a  score  of  1  and  the  fastest  team  will  have  the  highest  score  (i.e.,  the  most  minutes  finished 
before  the  slowest  team). 

4.2.3.  Findings 

The  results  from  the  laboratory  behavioral  simulations  address  the  technical  objectives  outlined 
in  section  4.2. 1  as  described  in  the  following  paragraphs. 

The  first  and  second  objectives  were  to  explore  the  way  in  which  individually-held  mental 
models  converge  among  team  members  to  become  shared  and  to  examine  different  means  of 
measuring  MMC.  Two  measurement  approaches  have  been  undertaken  to  address  these 
objectives.  First,  the  use  of  interrater  agreement  to  assess  the  degree  of  convergence  was  tested. 
The  commonality  across  team  member’s  mental  model  content  provides  a  means  of  measuring 
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MMC.  Commonality  may  range  along  a  continuum  from  no  commonality  in  mental  model 
content  to  identical  content,  with  most  team  members  having  some  degree  of  overlap  between 
the  two  extremes.  Interrater  agreement  is  a  commonly  used  statistic  to  assess  overlap  of  this 
type.  In  particular,  we  apply  Lindell  and  colleagues’  (1999)  r'wg{J)  as  it  provides  an  index  of 

interrater  agreement  for  multi-item  scales  such  as  the  ones  we  used  to  collect  data  regarding  the 
mental  models  of  interest.  To  follow  the  paradigm  that  mental  models  are  held  at  the  individual 
level,  we  separated  this  team-level  statistic  into  the  individual  contributions  from  each  team 
member  to  the  team’s  interrater  agreement  score.  This  separation  allowed  us  to  assess  (1)  the 
convergence  of  the  individually  held  mental  models  over  time  and  (2)  their  impact  on  team 
performance.  To  examine  this  approach,  we  analyzed  the  unrestricted  data  set.  The  results  in 
Table  2  demonstrate  the  applicability  of  this  technique  for  the  set  of  teamwork  mental  models 
tested.  The  numbers  in  the  body  of  the  table  represent  how  convergent  each  individual’s  mental 
models  are  with  her/his  team  mate’s  mental  models.  In  each  case,  the  degree  of  convergence 
increases  from  the  first  session  to  the  second.  Thus,  this  approach  demonstrates  that  mental 
models  converge  over  time  in  answer  to  the  first  objective  and  is,  therefore,  a  viable  measure  of 
mental  model  convergence  in  answer  to  the  second  objective. 

Table  2.  Results  of  paired  t-test 


Time  1 

Time  2 

/-value 

Goal  acceptance  mental  model 

0.75  (0.24) 

0.80  (0.25) 

=2aT 

Goal  clarity  mental  model 

0.69  (0.30) 

0.75  (0.28) 

-2.28* 

Membership  mental  model 

0.53  (0.38) 

0.67  (0.33) 

-4.17*** 

Cooperation  mental  model 

0.75  (0.37) 

0.83  (0.24) 

-4.35*** 

M  ( SD )  reported. 

+p  <  .10,  *p  <  0.05,  **p  <0.01,  ***p  <  0.001 


The  second  measurement  approach  used  to  watch  convergence  unfold  over  time  and  measure  the 
degree  of  convergence  is  process  tracing,  a  qualitative  analysis  of  transcript  data.  A  coding 
scheme  was  devised  by  the  principle  investigator  and  a  doctoral  student  who  had  worked 
extensively  on  this  project.  Two  graduate  students  unfamiliar  with  the  study  hypotheses  are 
currently  coding  the  transcripts.  Based  on  the  coding  scheme,  they  are  looking  to  code  three 
dimensions.  First,  they  are  identifying  topic  changes  and  completed  conversations.  Second,  they 
are  categorizing  statements  as  they  relate  to  one  of  six  different  mental  model  contents:  (1) 
approach-how  the  team  plans  to  attack  the  task;  (2)  goal-the  objective  of  the  assigned  task;  (3) 
member-participant  skills  that  may  or  may  not  be  related  to  the  task;  (4)  rules-roles  assigned  to 
each  participant  (i.e.,  union  representative,  human  resource  manager,  production  manager),  rules 
associated  with  each  role,  penalties  associated  with  breaking  rules,  and  schedule  requirements; 
(5)  task-when  the  team  discusses  actual  work  being  done  to  complete  the  task;  and  (6)  work 
allocation-how  the  team  members  plan  to  distribute  the  tasks  among  themselves.  Third,  they  are 
coding  the  type  of  exchange  that  is  taking  place:  (1)  action-someone  stating  that  they  are  doing 
something  or  directing  someone  else  to  do  something;  (2)  agree-statement  of  agreement  or 
confirmation;  (3)  apply-statement  applying  a  rule  or  an  approach  the  team  members  have  agreed 
upon;  (4)  clarify-questions  or  questioning  statements  about  team  activity;  (5)  information- 
statement  of  fact  that  passes  information  to  the  rest  of  the  team;  (6)  suggest-statement  suggesting 
a  course  of  action;  and  (7)  summary-statement  integrating  more  than  one  person’s  input.  By 
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using  the  process  tracing  methodology,  we  will  be  able  to  identify  the  degree  of  detail  shared 
among  team  members  that  lead  to  effective  team  performance  in  answer  to  the  first  objective  and 
patterns  associated  with  mental  model  convergence  in  answer  to  the  second  objective.  Tables  3 
and  4  provide  coded  transcript  excerpts  from  two  CM  teams  (Teams  28  and  30)  as  examples  of 
these  two  phenomena.  Team  30  is  a  higher  performing  team  (i.e.,  their  quality  score  was  higher 
than  Team  28’s  score).  In  Table  3,  Team  28  shares  details  of  the  rules  that  govern  their 
behavior,  whereas  Team  30  simply  identifies  that  they  each  have  different  rules.  Team  28 
spends  considerable  time  during  their  session  revisiting  the  rules  in  attempts  to  clarify  (well 
beyond  the  discussion  included  in  the  excerpt).  Team  30  doesn’t  speak  about  the  rules  in  any 
detail  for  the  remainder  of  their  session.  Table  4  provides  a  sample  showing  how  Team  30’s 
mental  models  converged  regarding  their  approach  to  the  task.  While,  conclusions  cannot  be 
drawn  based  on  these  samples,  the  power  of  process  tracing  to  uncover  the  way  in  which  mental 
models  converge  is  demonstrated. 

The  third  and  fourth  objectives  were  to  distinguish  among  the  multiple  mental  models  working 
simultaneously  within  a  team  and  to  determine  how  shared  mental  models  regarding  teamwork 
impact  team  performance  over  time.  In  Table  5  the  results  of  performance  regressed  on  a  set  of 
control  variables  and  the  teamwork  mental  model  convergence  scores  are  presented.  As  can  be 
seen  from  these  results,  the  four  teamwork  mental  models  examined  have  differing  relationships 
with  the  three  performance  measures  tested,  thereby  confirming  that  multiple  mental  models  are 
functioning  simultaneously  on  teams. 

The  results  in  Table  5  also  allow  us  to  examine  the  relationship  between  teamwork  mental 
models  and  team  performance  (the  fourth  objective).  The  speed  with  which  the  team  is  able  to 
complete  its  task  during  the  initial  stage  of  team  interaction  (i.e.,  session  1)  is  augmented  by  a 
common  understanding  of  the  importance  of  its  goal  and  a  corresponding  commitment  to  work 
towards  it  (the  goal  acceptance  mental  model)  and  a  shared  feeling  that  team  members  could 
collectively  work  together  to  accomplish  the  task  (the  cooperation  mental  model).  In  the  second 
session,  an  agreement  about  the  cooperative  nature  of  team  interaction  was  still  important,  but, 
with  respect  to  the  team’s  goal,  too  much  agreement  about  the  degree  of  clarity  regarding  the 
assignment  (the  goal  clarity  mental  model)  degraded  performance.  This  result  is  not  surprising 
given  that  for  this  set  of  data,  the  time  to  complete  the  task  was  not  restricted  in  any  way.  Thus, 
teams  who  shared  the  common  goal  of  getting  the  best  solution  in  order  to  win  the  prize  money 
spent  more  time  completing  the  task.  No  teamwork  mental  models  impacted  the  quality  of  the 
team’s  solution  during  the  first  session.  In  the  second  session,  a  commonly  held  belief  that  the 
team  was  staffed  with  members  able  to  complete  the  task  (the  membership  mental  model) 
resulted  in  a  strongly  positive  relationship  with  team  performance.  Finally,  we  examined  the 
relationships  between  teamwork  mental  models  and  team  member  satisfaction.  In  the  initial 
phases  of  team  interaction,  a  common  understanding  with  her/his  team  mates  about  the  clarity 
regarding  the  assignment  and  agreement  regarding  the  cooperative  work  environment  led  to 
higher  individual-level  satisfaction  with  the  team.  The  cooperation  mental  model  continued  to 
be  an  important  indicator  of  satisfaction  during  the  second  session. 

Several  other  interesting  findings  have  been  uncovered  through  data  analysis.  These  results  are 
highlighted  in  the  journal  article,  conference  paper,  and  dissertation  summaries  included  in 
section  7.0  of  this  report. 
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Table  3:  Differing  levels  of  detail  in  the  rules  mental  model 


Team  28  Time  1 


rules 

information 

a:  and  make  sure  each  works  the  30  hours  a  week  and  atleast4  a  shift 

im 

a:  it's  at  the  bottom  of  the  blue  sheet 

information 

a:  the  requirments 

finish 

rules 

summary 

b:  Okay,  our  instructions  are  different 

IjjfSiBliEEjft1 

rules 

information 

c:  it  says  employees  canwork  10/day  and  50  a  week  before  the  go  overtime 

information 

b:  mine  doesn't  say  anything  about  thirty  hours 

agree 

b:  okay 

isaMH 

a:  really 

c:  whats  on  your  nick? 

information 

b:  how  much  we  get  penalized 

c:  no  time  Imt  or  anything  ? 

information 

a:  yeah  1 .8  right 

information 

b:  it's  complicated  to  explain 

agree 

agree 

c:  yeah  thats  teh  overtime 

information 

a:  it's  not  over  time  it's  the  union  contract  requirements 

clarify 

b:  so  our  guidelines  are  less  than  10  hours  a  day  per  person  and  less  than  50 
/week  per  person?? 

information 

c:  all  mine  says  is  that  employees  can  only  work  10/day  or  50/weke  before  the 

penalties 

information 

a:  no  at  lease  30  hours  per  day 

a:  no  per  week 

clarify 

b:  and  did  1  hear  that  we  have  30 

agree 

b:  okay  nice 

agree 

b:  thanks 

finish 

wsmm 

summary 

c:  so  atleast  30  not  more  than  50  per  week 

finish 

summary 

a:  and  if  they  are  schedualed  to  work  they  have  to  work  at  lease  4  hours  a  day 

agree 

b:  beautiful,  lets  do  it. 

agree 

a:  yeah 

agree 

c:  ok  cool 

Team  30  Time  1 


rules 

EUSSBUHl 

a:  can  emplaoyees  whork  more  then  one  shift  in  the  day 

K2Uii>'K$& 

b:  yes 

agree 

a:  aight 

information 

information 

a:  my  papaer  did  not  say  that 

b:  oh  look  at  the  bottom  paragraph 

information 

a:  i  am  the  production  manager 

information 

b:  oh  i  am  the  human  resource  manager 

summary 

a:  mustt  be  differenrt 

information 

c:  no  less  than4  hrsandno  more  than  30hrs 

b:  todd  what  are  u? 

EEBSflMHi 

c:  confused 

a:  we  all  have  different  parts  and  have  to  work  as  a  team 

ESEESSHH 

c:  1  have  houly  wages...  do  you? 

KSEESHm 

a:  yeah 

summary 

b:  ok  so  todd  is  no  less  than  4  per  day  and  mine  is  n  more  than  10  per  day 

information 

b:  i  have  workforce  wages 

E3MMBE1 

b:  i  think  its  the  same 

a:  thats  the  same  thing  miss  SOM 

finish 

b:  ok 
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Table  4:  Approach  mental  model  convergence 


Team  30  Time  1 


approach 

c:  1  say  we  fill  up  the  days  more  employees  are  needed  like  day  5  and  6 

i _ 

b:  ok  so  howabout  ABC  work  10  hrs  day  6 

b:  and  only  4  hrs  on  day  7 

ciEm-ai 

task 

^Fir'kVsm 

c:  how  many  hrs  is  that  total? 

pJmMi 

b:  30  if  leave  day  3,  5,  6  and  2  shifts  on  day  7 

b:  for  ABC 

agree 

c:  ok 

agree 

c:  that  will  work 

®h?!?!=5^bbi 

b:  so  remove  them  from  shift  12  to  4  on  day  7  nick 

a:  we  need  to  clear  it  all  i  think 

agree 

b:  ok  wait  one  sec 

b:  ok 

ESilsSSBHi 

a:  lets  go  through  and  max  out  each  emoployee  one  at  a  time 

task 

|p"l  ill  liBtiUM 

b:  ok  so  clear  all 

a:  is  that  ok  with  todd 

c:  ok 

agree 

EjEESSHHI 

rules 

a:  wasn't  it  said  that  an  employee  can  max  out  at  50  hrs  ??? 

■nmisnfca 

b:  for  me  but  not  for  todd 

c:  not  with  me. .its  only  30  with  me 

information 

b:  todd  is  30  hrs  per  week 

.r(' 

■ 

a:  ok 

b:  so  we  will  max  out  ABC 

unrelated 

c:  1  have  1  minute  left  on  my  clock 

task 

information 

a:  a  is  maxed  out 

action 

b:  ok  so  put  B  and  C  with  him 

agree 

a:  ok 

agree 

c:  good 

Table  5.  Results  of  regression  analyses 


Time 

Performance 

Quality 

Performance 

Team  Member 
Satisfaction 

Time  1 

Time  2 

Time  1 

Time  2 

Time  1 

Time  2 

Control  variables 

Time  performance  Time  1 

n.a. 

0.21** 

n.a. 

n.a. 

n.a. 

n.a. 

Quality  performance  Time  1 

n.a. 

n.a. 

n.a. 

0.40*** 

n.a. 

n.a. 

Member  Satisfaction  Time  1 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

0.05 

Communication  medium 

0.27 

0.28*** 

0.09 

0.14** 

0.12 

0.19** 

Team  member  ability 

-0.01 

0.02 

-0.02 

-0.03 

-0.04 

-0.02 

Main  effect 

Goal  acceptance  mental  model 

0.26 

-0.06 

-0.05 

0.07 

0.11 

0.07 

Goal  clarity  mental  model 

-0.10 

-0.15* 

0.06 

-0.03 

0.15* 

-0.06 

Membership  mental  model 

0.07 

0.06 

0.10 

0.33*** 

0.11 

-0.03 

Cooperation  mental  model 

0.33** 

0.21** 

0.07 

-0.07 

0.20** 

0.34*** 

F- value 

16.7*** 

8.6*** 

1.5 

13.9*** 

9.4*** 

6.4*** 

R2 

0.33 

0.23 

0.04 

0.32 

0.21 

0.18 

R2- adjusted 

0.31 

0.20 

0.01 

0.30 

0.19 

0.15 

Standardized  estimates  reported. 
Experimental  condition:  0  =  CM;  1  =  FF. 
+p  <  0.10,  *p  <  0.05,  **p  <0.01,  ***p  <  0.001 
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5.0  DELIVERABLES 

In  addition  to  the  journal  articles  and  conference  papers  discussed  in  section  7.0,  two  products 
resulted  from  this  project.  The  first  is  the  means  of  measuring  mental  model  convergence 
through  interrater  agreement.  The  twg  and  individual  contributions  to  twg  could  be  used  in  real¬ 
time  to  assess  team  mental  model  convergence  and  identify  any  individual(s)  who  need  to  revise 
their  individually  held  mental  models  to  increase  the  degree  of  convergence. 

The  second  tool  is  the  Decision  tree  for  Information  SHaring  (DISH)  tool.  The  study  results 
have  been  mapped  to  a  decision  tree  to  help  collaborators  focus  on  (1)  the  information  that  needs 
to  be  shared  and  (2)  the  processes  we  found  to  be  most  effective  under  different  collaborative 
conditions.  The  DISH  tools  for  FF  and  CM  teams  are  presented  in  Figures  6  and  7,  respectively. 
The  first  decision  relates  to  the  conditions  under  which  the  team  is  working  (i.e.,  unrestricted, 
time  pressure,  environmental  uncertainty).  The  second  decision  relates  to  the  amount  of  time  the 
team  has  been  working  together  (i.e.,  inexperienced  or  experienced).  Finally,  the  type  of 
performance  the  team  wants  to  optimize  (i.e.,  time  or  quality  performance)  must  be  specified. 


18 


6.0  IMPACT 

The  results  of  this  research  impact  practitioners  and  academicians.  Practitioners  can  use  the 
results  to  guide  teams  as  they  work  to  achieve  the  appropriate  levels  of  cognitive  unification. 
Managers  in  organizations  using  cross-functional  project  teams  may  find  a  number  of  useful 
strategies  in  this  research,  because  an  understanding  of  mental  model  content  and  the  mental 
model  convergence  process  should  improve  the  effectiveness  of  project  teams.  First,  knowledge 
of  the  specific  mental  model  contents  and  the  need  to  actively  think  about  these  issues  may  be 
communicated  to  newly  formed  cross-functional  project  teams  to  help  them  focus  their  attention 
on  teamwork  issues.  By  discussing  the  issues  that  should  be  actively  addressed,  managers  and 
team  leaders  can  help  to  expedite  and  manipulate  the  way  in  which  shared  mental  models 
develop.  Explicitly  developing  shared  mental  models  should  create  a  solid  foundation  upon 
which  taskwork  can  be  accomplished  more  expeditiously.  Further,  team  trainers  may  devise 
training  programs  focused  on  improving  teamwork  through  the  development  and  maintenance  of 
shared  mental  models  based  on  this  work.  Specifically,  the  results  of  this  research  can  be  used  to 
develop  learning  aids  that  will  help  teams  create  shared  mental  models  at  the  onset  of  their  life 
cycle  and  diagnostic  tools  to  pinpoint  areas  for  intervention  when  existing  teams  are  not 
performing  effectively.  The  creation  and  validation  of  these  tools  is  an  opportunity  for 
practitioners  and  academicians  to  collaborate. 

Researchers  interested  in  learning  more  about  how  teams  can  function  more  effectively  may  also 
find  value  in  this  work.  In  general,  exploring  the  cognitive  aspects  of  teamwork,  through  MMC, 
may  provide  insight  into  how  team  members  can  make  the  necessary  shift  in  their  perspectives 
from  the  individual  to  the  team.  Further,  increasing  our  understanding  of  the  cognitive  processes 
associated  with  effective  teamwork  facilitates  developing  theoretical  explanations  for  the 
respective  success  and  failure  of  seemingly  similar  teams.  Finally,  this  research  advances  the 
team  cognition  literature  by  integrating  various  extant  literatures  and  studying  empirically  the 
model  of  MMC. 

This  research  project  relates  specifically  to  project  teams.  They  are  not,  however,  the  only  team 
type  that  can  benefit  from  this  research.  The  type  of  team  will  dictate  the  appropriate  content  of 
the  mental  models.  The  content  requirements  will,  therefore,  be  unique  for  each  type  of  team 
under  investigation.  The  development  process,  however,  will  be  consistent  across  team  type. 
Regardless  of  team  type,  team  members  must  be  able  to  orient  themselves  to  the  team  and  task, 
differentiate  among  themselves  and  integrate  their  various  perspectives  to  create  and  sustain  fully 
functioning  SMM. 
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7.0  PUBLICATIONS 

To  date,  two  conference  papers  have  been  presented,  one  conference  paper  has  been  accepted  for 
presentation,  one  journal  article  has  been  conditionally  accepted,  one  dissertation  has  been 
completed,  three  journal  articles  are  in  preparation  (submission  of  all  three  is  expected  by  year 
end),  five  additional  journal  articles  and  one  additional  conference  paper  are  planned.  The 
publications  resulting  from  this  research  effort  are  summarized  on  the  following  pages. 


7.1.  Listing  of  Journal  Article  Summaries 


Page 

Mental  model  convergence:  A  conceptual  framework  and  preliminary  evidence . 23 

Capturing  the  convergence  of  multiple  mental  models  and  their  impact  on  team  performance  ...24 
Different  time-tables:  Understanding  processes  in  computer-mediated  and  face-to-face  teams.. 25 


Incorporating  human  behavior:  A  framework  for  modeling  and  simulation . 27 

Planned  journal  articles . 29 


7.2.  Listing  of  Conference  Paper  Summaries 

Enhancing  safety  in  the  event  of  catastrophic  failure:  The  case  for  continual  use  systems . 30 

A  framework  for  social  system  modeling  and  simulation . . . 31 

Accomplishing  globally  distributed  work:  A  case  for  helping  virtual  teams  with  their  taskwork 
processes . 32 

Planned  conference  paper . 33 

7.3.  Listing  of  Dissertation  Summary 

Modeling  teams:  A  general  systems  theory  approach  . 34 
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Manuscript  Title: 

Mental  model  convergence:  A  conceptual  framework  and  preliminary  evidence 

Author: 

McComb,  S.A. 

Publication  Status: 

Manuscript  conditionally  accepted  for  publication  in  Research  on  Multi-Level  Issues. 

Research  Question: 

What  is  the  mental  model  convergence  process? 

Summary: 

This  manuscript  presents  the  conceptual  development  of  the  process  by  which  the  content  of 
individually  held  mental  models  converges  over  time.  An  overview  of  this  process  is  presented 
in  section  3.0  of  this  report.  Additionally,  the  results  of  the  quasi-experiment  (discussed  in 
section  4.1.3)  are  presented  as  preliminary  evidence  of  the  convergence  process.  Moreover, 
these  results  highlight  the  influence  of  converged  mental  models  on  team-level  performance. 
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Manuscript  Title: 

Capturing  the  convergence  of  multiple  mental  models  and  their  impact  on  team  performance 

Authors: 

McComb,  S.A.  and  Vozdolska,  R.P. 

Publication  Status: 

Manuscript  in  preparation  for  submission. 

Hypotheses: 

[HI]  Interrater  agreement  will  depict  mental  model  convergence. 

[H2]  Multiple  mental  models  will  exist  simultaneously. 

[H3]  Shared  mental  models  about  the  team  and  teamwork  processes  will  positively  influence 
team  performance. 

The  results  of  this  manuscript  were  presented  in  section  4.2.3. 
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Manuscript  Title: 

Different  time-tables:  Understanding  processes  in  computer-mediated  and  face-to-face  teams 


Authors: 

Vozdolska,  R.P.  and  McComb,  S.A. 


Publication  Status: 

Manuscript  in  preparation  for  submission. 

Hypotheses: 

[HI]  The  differences  among  processes  in  CM  and  FF  teams  will  decrease  after  the  second 
performance  episode. 

[H2]  The  process  effects  on  performance  in  CM  and  FF  teams  will  be  more  similar  after  the 
second  performance  episode  than  after  the  first. 


Results: 

The  unrestricted  experiments  conducted  provided  data  for  this  study.  The  two  performance 
episodes  referenced  in  the  hypotheses  are  session  1  and  session  2  of  the  behavioral  simulation. 
The  experimental  results  are  shown  in  Tables  6  and  7. 

The  results  in  Table  6  indicate  that  when  the  teams  completed  their  performance  episodes,  they 
did  not  differ  significantly  in  their  quality  performance  achieved,  so  the  processes  they  used  can 
be  compared  directly.  The  results  from  the  t-tests  support  our  first  hypothesis.  In  the  first  time 
period  all  four  processes  are  significantly  different.  In  the  second  performance  episode  two  of  the 
four  processes  are  essentially  the  same,  showing  that  the  processes  used  by  CM  and  FF  teams 
started  to  converge. 


Table  6:  T-test  comparisons  between  FF  and  CM  teams 


Dependent  Variable 

Face-to-face 

Computer-Mediated 

t-Value 

Mean 

St.Dev 

Mean 

St,Dev 

0.958 

0.04 

0.94 

0.08 

-1.06 

Quality  Performance  (T2) 

0.983 

SSsEEHS 

0.97 

0.05 

-1.84 

Collective  Efficacy  (Tl) 

4.28 

HKEEKll 

3.77 

0.64 

-3.44*** 

1.86 

0.51 

2.15 

0.55 

2.31* 

Decision  Making  (Tl) 

4.39 

0.54 

4.04 

0.65 

-2.51** 

Coordination  (Tl) 

3.92 

0.61 

3.57 

0.70 

-2.21* 

4.50 

0.42 

4.21 

0.59 

-2.48* 

Conflict  (T2) 

1.77 

0.47 

1.96 

0.71 

1.34 

4.40 

0.41 

4.15 

0.62 

-2.05* 

4.26 

0.49 

4.07 

0.58 

-1.50 

*p  <0.05,  **  p<  0.01,  ***  p<0.001 


The  results  in  Table  7  support  our  second  hypothesis.  During  the  first  performance  episode  the 
effects  on  performance  are  very  different.  For  FF  teams  none  of  the  processes  studied  has  an 
effect  on  performance,  while  for  the  CM  teams  collective  efficacy  and  coordination  are 
significant  predictors.  During  the  second  performance  episode  both  regressions  are  significant, 
the  only  difference  is  that  decision  making  is  marginally  significant  for  CM  teams  and  non¬ 
significant  for  the  FF  teams. 
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Table  7:  Results  of  Regression  Analysis  for  Team  Processes  Time  1  and  Time  2  (N=72) 


Variable 

Face-to-Face  Teams 

Quality  Performance 

Computer-Mediated  Teams 

Quality  Performance 

Time  1 

Time  2 

Time  1 

Time  2 

Time  Performance  (Tl) 

0 

0 

Major 

0.008 

-0.02 

Years  Spent  in  College 

-0.003 

-0.02 

Max  Grade  per  Team 

0.008 

-0.02 

Collective  Efficacy  (Tl) 

-0.011 

0.11** 

Conflict  (Tl) 

-0.007 

-0.02 

■aamn 

-0.007 

-0.01 

Coordination  (Tl) 

0.005 

-0.06* 

Adi.  R2 

0 

0.24* 

Quality  Performance  (Tl) 

0.12 

0.21* 

Time  Performance  (T2) 

0 

-0.001 

Major 

0.006 

-0.02 

-0.002 

-0.02 

Max  Grade  per  Team 

0.01** 

-0.01 

Collective  Efficacy  (T2) 

-0.009 

0.02 

Conflict  (T2) 

-0.02** 

-0.03* 

mammammsm 

giiinrf 

-0.05* 

hw»f,wv.iiiwif.r,iiwi 

0.04 

Adj.  R2 

0.46** 

0.29* 

*  p  <0.05,  **  p<  o.oi,  ***  p<0.001 


Conclusions: 

The  results  from  these  analyses  show  that  CM  and  FF  team  processes  should  eventually 
converge.  Given  that  the  CM  teams  had  enough  time  to  get  comfortable  with  their  leaner 
medium,  they  are  able  to  establish  the  same  processes  as  the  FF  teams.  This  result  will  allow 
researchers  and  practitioners  to  use  the  vast  extant  literature  on  team  performance  (comprised 
primarily  of  research  conducted  on  FF  teams)  as  a  baseline  for  working  with  CM  teams  rather 
than  devising  a  new  research  stream  for  what  some  consider  to  be  a  unique  team  type. 
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Manuscript  Title: 

Incorporating  human  behavior:  A  framework  for  modeling  and  simulation 

Authors: 

Vozdolska,  R.P.  and  McComb,  S.A. 

Publication  Status: 

Manuscript  in  preparation  for  submission. 

Research  Question: 

What  is  an  effective  approach  for  constructing  quantifiable  models  of  interdependent  social 
system  behavior? 

Results: 

An  eight  step  framework  for  constructing  quantifiable  social  behavior  models  is  theoretically 
developed.  The  steps  are  presented  in  Figure  9  of  this  report  when  the  conference  paper  about 
this  framework  is  presented.  Each  step  is  based  on  existing  literature.  The  framework  is 
illustrated  with  a  model  of  team  communication  presented  parallel  to  the  discussion  of  each  step. 
The  team  communication  model  demonstrates  the  applicability  of  the  framework  proposed  to 
social  science  research. 

Models  developed  using  this  framework  can  be  simulated  (for  our  research,  we  used  Matlab  to 
simulate  the  model).  In  Figure  8,  we  present  a  subset  of  the  simulations  performed  to 
demonstrate  the  result’s  usefulness.  These  results  highlight  how  we  can  compare  the  behavior  of 
systems  with  varying  numbers  of  internal  states,  allow  us  to  draw  conclusions  about  the 
modeled  system’s  behavior  from  the  overall  form  of  the  relationships,  the  relative  order  in  which 
the  state  variables  appear  compared  to  each  other,  and  the  absolute  value  for  each  variable  at  a 
specific  time. 

Figure  8:  System  performance  samples 
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Conclusions: 

We  advocate  two  main  requirements  for  model  development.  First,  the  model  must  be 
constructed  precisely.  Second,  the  model  must  be  based  on  existing  evidence.  Indeed,  the  model 
must  be  strictly  grounded  in  empirical  research  and,  ultimately,  tested  against  some  known 
outcomes;  otherwise  the  results  obtained  may  not  be  predictive  of  true  system  behavior. 

The  framework  we  presented  helps  bridge  the  gap  between  qualitative  and  quantitative 
paradigms  in  two  distinct  ways.  First,  researchers  can  augment  a  model  that  traditionally  would 
not  have  a  human  component,  such  as  a  project  management  model,  with  a  second, 
complementary  model  of  social  behavior,  such  as  our  model  of  team  communication. 
Alternatively,  human  behavior  can  be  incorporated  into  system  models  directly,  such  as  adding 
the  functional  expertise  of  the  team  to  the  aforementioned  project  management  model.  When 
constructs  are  to  be  incorporated,  using  the  framework  can  aid  the  developer  in  ensuring  that  the 
behavior  variables  are  operationalized  appropriately. 
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Planned  Journal  Articles 


[1]  Examining  how  mental  model  convergence  unfolds  over  time.  Transcripts  of  the 
behavioral  simulation  sessions  will  be  analyzed  using  process  tracing  to  look  for  (a)  patterns 
in  the  mental  model  convergence  process  and  (b)  the  level  of  detail  among  each  individual 
team  member’s  converged  mental  models  (i.e.,  are  individual  mental  models  completely 
overlapping  or  do  individuals  share  only  high  level  details).  This  analysis  will  be  used  to 
better  understand  the  relationship  between  team  performance  and  both  the  convergence 
process  and  the  level  of  detail  included  in  converged  mental  models. 

[2]  Exploring  the  effects  of  time  pressure  and  environmental  uncertainty  on  mental  model 
convergence.  The  aforementioned  study  of  mental  model  convergence  measured  as 
interrater  agreement  will  be  replicated  using  the  data  from  the  time  pressure  and 
environmental  uncertainty  conditions. 

[3]  Understanding  processes  in  computer-mediated  and  face-to-face  teams  dealing  with  time 
pressure  and  environmental  uncertainty.  The  aforementioned  study  comparing  the 
processes  used  by  CM  and  FF  teams  will  be  replicated  using  data  from  the  time  pressure  and 
environmental  uncertainty  conditions. 

[4]  Computer-mediated  and  face-to-face  team  behavior  comparison.  The  two  models 
developed  in  Patrashkova’s  dissertation  will  be  presented  and  the  conclusions  about  team 
behavior  made.  The  comparison  between  CM  and  FF  teams  shows  that  CM  teams  experience 
more  difficulties  in  establishing  patterns  in  their  behavior.  More  team  processes  fluctuate 
than  in  FF  teams.  Face-to-face  teams  are  more  satisfied  with  the  team  and  are  able  to 
establish  stable  team  behavior  more  quickly. 

[5]  Exploring  team  behavior.  The  models  from  Patrashkova’s  dissertation  will  be  explored  in 
terms  of  sensitivity  to  initial  conditions,  stability,  equifinality  and  controllability  of  the 
systems. 

[6]  Determining  the  effects  of  team  member  personality  on  team  conflict  and  performance. 
Each  team  member  completed  a  personality  questionnaire.  This  data  will  be  used  to  study 
mediating  effects  of  team  conflict  on  the  relationship  between  member  personality  and  team 
performance. 


Conference  Paper  Title: 

Enhancing  safety  in  the  event  of  catastrophic  failure:  The  case  for  continual  use  systems 

Authors: 

McComb,  S.A.  and  Deshmukh,  A.V. 

Conference: 

First  International  Conference  on  Safety  and  Security  Engineering,  Rome,  Italy,  13-15  June 
2005. 

Publication: 

Manuscript  will  be  published  by  Wessex  Institute  of  Technology  in  a  forthcoming  volume. 

Research  Question: 

What  is  the  most  effective  system  design  for  response  systems? 

Results: 

The  experiments  conducted  with  environmental  uncertainty  provided  data  for  this  study.  The 
experimental  results  are  shown  in  Table  8.  Each  team’s  experimental  session  was  reviewed  to 
determine  if  they  developed  a  routine  for  solving  the  task  and  if  they  were  able  to  recover  after 
the  interruption.  We  compared  the  performance  of  (1)  those  with  and  without  routines,  (2)  those 
who  recovered  with  those  who  did  not  recover,  and  (3)  those  that  had  a  routine  and  were  able  to 
recover  with  those  that  did  not  exhibit  both  behaviors.  As  can  be  seen  in  the  table,  teams  with 
routines  (p=0.12)  and  teams  able  to  recover  (p=0.07)  perform  more  effectively  than  those  in  the 
opposite  condition.  Teams  meeting  both  conditions,  however,  demonstrated  the  most  significant 
difference  between  conditions  (p=0.05).  Thus,  our  results  indicate  that  both  an  established 
routine  and  the  ability  to  invoke  it  after  an  environmental  disturbance  are  critical  for  a  team’s 
ability  to  perform  at  the  highest  levels. 

Table  8:  Study  results 


Routine 

Recovery 

Routine  and 
Recovery 

Yes 

No 

Yes 

No 

Yes 

No 

N 

53 

13 

43 

23 

41 

25 

Mean 

3126 

3229 

3103 

3226 

3098 

3225 

SD 

200 

267 

148 

293 

145 

285 

p-value 

0. 

12 

0. 

97 

0. 

95 

Note:  Low  scores  indicate  better  performance. 


Conclusions: 

The  results  of  this  study  were  used  to  build  a  case  for  the  design  and  implementation  of  continual 
use,  robust  systems  that  can  be  used  during  normal  operations  and  function  effectively  during 
disruptions  as  an  alternative  means  of  preparing  for  extreme  event  failures.  In  the  manuscript, 
we  present  an  example  of  an  information  infrastructure  that  allows  the  realization  of  such 
continual  use  processes  and  highlight  opportunities  for  further  research. 
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Conference  Paper  Title: 

A  framework  for  social  system  modeling  and  simulation 

Authors: 

Patrashkova,  R.P.  and  McComb,  S.A. 

Conference: 

Academy  of  Management,  Honolulu,  Hawaii,  5-10  August  2005. 

Publication: 

Abstract  published  in  the  Academy  of  Management  Proceedings. 

Research  Question: 

How  can  a  social  system  be  represented  as  a  rigorous  mathematical  formalization? 

Results: 

A  framework,  comprising  the  steps  presented  in  Figure  9  is  developed.  The  theoretical 
background  for  each  step  is  thoroughly  presented.  Next,  its  application  to  social  science  models 
is  discussed  and  last,  each  step  is  illustrated  by  building  a  model  of  team  communication. 

Figure  9:  Framework  for  devising  a  rigorous  mathematical  formalization 


Conclusions: 

Our  framework  builds  upon  systems  theory  and  social  science  research.  A  systems-based  social 
science  model  will  represent  the  total  system  performance,  even  when  a  change  in  only  one  or  a 
few  of  its  parts  is  contemplated,  and  thus  may  help  us  better  understand  how  social  system 
behavior  unfolds  over  time. 

The  purpose  of  our  framework  is  to  help  future  researchers  avoid  the  epistemological 
complications  that  often  accompany  the  use  of  social  science  research  in  mathematical  models 
and  simulations.  Such  a  framework  can  help  researchers  without  extensive  mathematical  training 
to  develop  and  test  quantitative  models,  as  well  as  replicate,  empirically  verify  and  extend  the 
output  of  models  already  developed. 
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Conference  Paper  Title: 

Accomplishing  globally  distributed  work:  A  case  for  helping  virtual  teams  with  their  taskwork 
processes 

Authors: 

McComb,  S.A.  and  Vozdolska,  R.P. 

Conference: 

First  International  Conference  on  Management  of  Globally  Distributed  Work,  Bangalore,  India, 
28-30  December  2005. 

Research  Question: 

Will  the  processes  found  to  contribute  to  the  performance  of  FF  teams  be  the  same  for  CM 
teams? 

Results: 

The  second  session  of  the  unrestricted  experiment  provided  data  for  this  study.  The  multiple 
regression  results  are  shown  in  Table  9. 

Table  9:  Study  results 


Taskwork  Processes 

P 

Intercept 

123.50*** 

Time  Performance  T1 

0.03 

Team  Skills 

g  gq*** 

Communication 

-36.13*** 

Conflict 

-12.74*** 

Decision  Making 

-11.38** 

Work  Allocation 

11.63+ 

F 

R2 

14  74*** 
0.75 

+p=0.06,  *p  <0.05,  **p<  0.01,  ***p<0.001 


Communication,  conflict,  decision-making,  and  work  allocation  all  play  a  significant  role  in 
explaining  team  performance.  More  communication,  greater  intrateam  conflict,  and  including 
everyone  in  decision-making  all  resulted  in  longer  times  to  complete  the  scheduling  task.  Time 
performance  could  be  enhanced  by  equitably  allocating  the  necessary  work. 

Conclusions: 

Through  this  research  we  aimed  to  identify  the  taskwork  processes  that  support  and  degrade 
virtual  team  performance.  Our  results  indicate  that  taskwork  processes  do  impact  team 
performance.  Thus,  teams  should  be  given  training  about  and  facilitators  should  assist  teams  to 
achieve  effective  communication,  conflict  avoidance  and  resolution,  decision-making,  and  work 
allocation  skills.  For  example,  effective  communication  via  electronic  media  is  not  easy  for  all 
individuals.  Trainers  and  facilitators  must  focus  on  helping  team  members  scribe  concise, 
informative  messages  that  focus  on  quality,  not  quantity  of  communication. 
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Planned  Conference  Paper 

[1]  Examining  how  mental  model  convergence  unfolds  over  time.  Transcripts  of  the  CM  teams 
in  the  control  condition  will  be  analyzed  using  process  tracing  to  look  for  (a)  patterns  in  the 
mental  model  development  process  and  (b)  the  level  of  detail  among  each  individual  team 
member’s  converged  mental  models  (i.e.,  are  individual  mental  models  completely 
overlapping  or  do  individuals  share  only  high-level  details).  This  analysis  will  be  used  to 
better  understand  the  relationship  between  team  performance  and  both  the  development 
process  and  the  level  of  detail  included  in  converged  mental  models. 
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Dissertation  Title: 

Modeling  teams:  A  general  systems  theory  approach 

Author: 

Patrashkova-Vozdolska,  R.P. 

Research  Objectives: 

(1)  to  design  a  framework  that  will  facilitate  the  development  of  dynamic  social  systems  models; 

(2)  to  show  that  GST  is  applicable  to  team  modeling  and  (3)  to  introduce  the  understanding  of 
dynamic  social  system  behavior  that  can  be  achieved  though  GST. 

Results: 

The  purpose  of  this  doctoral  dissertation  is  to  show  that  models  exhibiting  complex  team 
behavior  can  be  developed  successfully  and  when  analyzed  provide  information  about  behavior 
that  cannot  be  obtained  with  conventional  statistics.  To  achieve  this  goal,  the  following  activites 
were  undertaken:  (1)  developed  a  framework  for  model  development  based  on  General  Systems 
Theory,  (2)  constructed  a  team  model,  derived  from  the  team  behavior  literature,  (3)  designed 
and  conducted  a  behavioral  simulation  to  collect  data  for  the  model  estimation,  (4)  estimated  the 
model  parameters  based  on  the  data,  (5)  simulated  the  behavior  of  the  modeled  system  using 
Matlab,  and  (6)  analyzed  the  simulation  model  to  evaluate  the  behavior  of  the  system. 

To  provide  data  for  the  model  the  72  teams  from  the  unrestricted  experiment  were  used.  Multiple 
regression  analyses  showed  that  all  hypothesized  relationships  are  significant,  except  Team 
Quality  Performance  was  the  dependent  variable.  Based  on  the  statistical  results,  simulation 
models  were  constructed  and  simulations  performed.  The  simulations  demonstrate  the  dynamics 
of  the  systems  studied.  Further,  the  simulated  systems  were  analyzed  to  assess  stability, 
equilibrium,  equifinality  and  controllability.  These  analyses  showed  that  the  systems  studied  are 
stable,  non-equifinal,  not  completely  controllable  and  do  not  achieve  equilibrium.  The  data  were 
split  and  separate  models  for  CM  and  FF  constructed.  To  demonstrate  the  usefulness  of  the 
simulated  output,  these  models  are  presented  in  Figure  10  and  allow  us  to  visually  compare  the 
dynamics  of  the  systems. 


Figure  10:  System  performance  samples  for  CM  and  FF  teams 
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Conclusions: 

The  aim  of  the  dissertation  research  was  to  develop  a  complex  dynamic  model  of  team  behavior. 
This  aim  was  achieved  and  the  model  was  developed,  verified  with  an  empirical  behavior 
simulation  and  analyzed  to  provide  expanded  team  behavior  understanding.  The  simulation 
output  allowed  us  to  see  the  dynamics  of  the  modeled  system,  compare  different  systems,  as  well 
as  evaluate  causality,  based  on  the  timing  of  events  in  the  simulation.  Further,  the  system  can  be 
diagnosed  and  inferences  about  structure  and  behavior  can  be  derived  when  the  GST  specific 
analyses  (e.g.  stability,  equifinality)  are  applied.  Last,  the  simulation  output  helps  determine 
when  a  change  may  occur  in  team  behavior  and  thus  can  help  with  planning  longitudinal 
research.  In  conclusion,  simulations  allow  us  to  see  and  estimate  system  behavior  dynamics  in  a 
way  that  complements  the  results  achieved  with  conventional  statistics. 
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